1. Field of the Invention
The invention generally relates to improving the contrast in digital images, and more specifically to an adaptive recursive filter which forms a pedestal signal from the original digital image. A tone scale function is applied to the pedestal signal and a texture signal is added to attain a processed digital image.
2. Description of the Related Art
It is well known that the dynamic range of photographic paper is less than the typical scene dynamic range. The result of this incongruity is that a good deal of scene content is rendered to black or white on the photographic print. For this reason, in an image-processing environment, a tone scale function may be used to reduce the scene dynamic range in order to map more information onto the display medium. There exist many processes for creating a tone scale function on an image dependent basis (e.g., see, U.S. Pat. No. 5,471,987 to Nakazawa et al. (hereinafter xe2x80x9cNakazawaxe2x80x9d), incorporated herein by reference). Each of the conventional tone scale function processes examines certain statistical characteristics of the image under consideration in order to automatically generate the tone scale function. In addition, tone scale functions may be generated with manual interactive tools.
After the tone scale function has been generated, there exists the question of how to apply the tone scale function to the digital image. The goal of dynamic range compression is to adjust the overall dynamic range of the image, rather than to affect the contrast of any given object in the image. In essence, tone scale functions meant to reduce the image dynamic range should be applied in such a way as to minimize the effect on the scene texture. This criterion excludes the possibility of applying the tone scale function directly to the image luminance channel. Thus, it is common to apply the tone scale function to a lower frequency sub-band of the image, preserving the higher frequency sub-band(s) that are considered to be image texture (e.g., see, U.S. Pat. No. 5,012,333, to Lee et al. (hereinafter xe2x80x9cLeexe2x80x9d) incorporated herein by reference).
As mentioned above, after the tone scale function has been generated, there exists the question of how to apply the tone scale function to the digital image. Application of a tone scale function meant for dynamic range compression directly to each color channel of an image results in desaturation. For this reason, it is a common practice to apply the tone scale function to a luminance (neutral) representation of the image. Direct application of the tone scale function to the image neutral channel tends to result in compression of detail in addition to compression of the overall image dynamic range, resulting in an image with a flat appearance.
Lee describes a procedure for preserving the high frequency detail of an image by blurring the image neutral channel in order to create a lowpass signal. Subtracting the lowpass signal from the image neutral channel produces a highpass signal. The processed image is generated by applying the tone scale function to the lowpass signal and adding the result to the high-pass signal. This procedure preserves a segment of the image frequency spectrum; however, artifacts are seen at large boundaries.
A. Gallagher and E. Gindele built on this work with U.S. Pat. No. 6,317,521, based on application Ser. No. 09/163,645, filed Sep. 30, 1998 (hereinafter xe2x80x9cGallagherxe2x80x9d; incorporated herein by reference). More specifically, Gallagher incorporated an artifact avoidance scheme along with a single standard FIR filter to generate the texture signal. Also, in U.S. Pat. No. 5,454,044, Nakajima suggests modifying the image contrast by the formula Sproc=Sorg+ƒ(Sus). In Nakajima incorporated herein by reference), the low frequency image Sus is passed through function ∂( ) which is a monotonically decreasing function. This signal is added to the original Sorg to create the processed image Sproc.
Another example is an FIR (finite impulse response) filter based process known as homomorphic filtering (e.g., see R. Gonzalez, R. Woods, Digital Image Processing, Addison-Wesley Publishing Company, New York, 1992, pp. 213-218, incorporated herein by reference), which modifies the low frequencies of an image to achieve a contrast modification. In homomorphic filtering, the high frequency information is again considered to be the image texture.
In U.S. Pat. No. 5,905,817, Matama (incorporated herein by reference) describes using an IIR (infinite impulse response) filter in essentially the same framework described by Lee. The advantage to this approach is speed. In addition, by using an IIR filter, the computational requirements remain constant despite any change to the desired level of blurring.
Each of these methods of applying a tone scale function to an image channel rely on a single blurring with a linear filter. Because of this, there is an inherent size selectivity property in the tone scale function application process. Image structures that are spatially smaller than a certain size are preserved, while details larger than that size are affected by the tone scale function. In addition, the preservation of high frequencies in an image may lead to the creation of unsharp mask type artifacts (overshoot and undershoot) in the neighborhood of large edges (characteristic of occlusion boundaries or dark shadows).
In general, it was observed that larger digital filters (used to create the lowpass signal) result is a more pleasing processed image, except for the fact that the artifacts become more objectionable. Thus, the goal is to achieve greater amounts of blur without producing the overshoot artifacts at edges. Several pyramid schemes have been developed in order to achieve this goal. Because the pyramid schemes consist of multiscale representations of the same image objects, the detail size range that is preserved may be modified throughout the image.
U.S. Pat. No. 5,467,404 to Vuylsteke et al. (incorporated herein by reference) describes a method of adjusting the coefficients of a wavelet pyramid in order to modify the contrast of the image while preserving details (and producing no artifacts). In U.S. Pat. No. 5,881,181 (incorporated herein by reference), Ito describes a general multi-resolution approach intent on achieving the same goals. These methods produce satisfactory results, but require a large number of filtering operations.
Another approach to the problem of tone scale function application is to use nonlinear filtering techniques that essentially preserve edges but blur out detail. In U.S. Pat. No. 5,796,870 (incorporated herein by reference), Takeo describes a large rectangular filter, long in the direction along an edge and short in the direction across an edge. This approach reduces the artifacts at edges, but diagonal edges pose a problem. Further, Nakazawa describes using an FIR filter whose weights are determined at each pixel location, based upon the absolute value of the difference of pixel intensities between two pixels falling under the digital filter. However, this method does not account for noise in the image, and is very time consuming.
None of the conventional methods discussed above allows for a relatively fast filtering means that preserves details (without requiring a specific detail size range). One drawback of conventional techniques is that direct application of the tone scale function to the image neutral channel tends to result in compression of detail in addition to compression of the overall image dynamic range, resulting in an image with a flat appearance. Further, generating the processed image by applying the tone scale function to the lowpass signal and adding the result to the high-pass signal preserves a segment of the image frequency spectrum; however, produces artifacts that can be seen at large boundaries.
There is an inherent size selectivity property in the tone scale function application process. Image structures that are spatially smaller than a certain size are preserved, however details larger than that size are adversely affected by the tone scale function. In addition, the preservation of high frequencies in an image may lead to the creation of unsharp mask type artifacts (overshoot and undershoot) in the neighborhood of large edges (characteristic of occlusion boundaries or dark shadows).
The relatively fast adaptive recursive filter described below allows the application of a tone scale function to a digital image to adjust the macro-contrast of the image, preserves detail without reference to a specific detail size range, and prevents artifacts in the neighborhood of large edges.
The invention comprises a process called the Adaptive Recursive Filter (ARF) tone scale function application process. With the inventive tone scale application process, the digital image channel is decomposed into pedestal and texture signals. The tone scale function is applied to the pedestal signal and the texture signal is added to obtain a processed digital image channel.
An important feature of the invention is the method implemented to generate the pedestal signal. More specifically, the invention includes a recursive filter that adaptively performs more blurring in relatively flat areas of the image and less blurring at large discontinuities.
The invention described below was developed to enable tone scale modification for dynamic range compression with reduced artifacts and fast implementation. More specifically, with the invention image decomposition is enabled by an adaptive recursive filter (ARF).
The invention inputs an image, divides the image into a pedestal signal and a texture signal, applies a tone scale function to the pedestal signal to produce a modified pedestal signal, and adds the texture signal to the modified pedestal signal to produce a processed digital image channel. The dividing filters a pixel of the image using weighting that is dependent upon coefficients of neighboring pixels adjacent the pixel. The filtering blurs the pedestal signal, such that flat areas of the image are blurred more than edges in the image.
The filtering is recursive filtering that includes four-pass recursive filtering, which includes forward filtering the digital image channel in a first direction, filtering the result of the first filtering stage in the reverse of the first direction, forward filtering the result of this second filtering stage in a second direction perpendicular to the first direction, and filtering the result of the third filtering stage in the reverse of the second direction.
The recursive filtering can also include performing a plurality of cascaded four-pass recursive filtering processes wherein an output of a previous four-pass recursive filtering process comprises an input for a next four-pass recursive filtering process. Alternatively, the recursive filtering could include performing a plurality of cascaded four-pass recursive filtering processes wherein an output of a first four-pass recursive filtering process comprises an input for all remaining four-pass recursive filtering processes.
The filtering includes calculating gradients of the neighboring pixels, finding a minimum gradient of the neighboring pixels, applying the minimum gradient to a look up table to output a first variable, determining a ratio of horizontal gradients to vertical gradients, and calculating the coefficients based on the first variable and the ratio.
The invention can also include a method of adjusting tone scale of a digital image that includes blurring a plurality of neighboring pixels from a digital image input signal, filtering a pixel of the neighboring pixels using weighting that is dependent upon coefficients of the neighboring pixels and generating a pedestal signal based on the filtering and blurring.
One advantage of the invention is that the tone scale function application process is relatively fast. Further, the image detail is preserved while the image macro-contrast is adjusted. In addition, the detail is preserved without reference to a specific detail size range (in contrast to most single linear filter approaches) and artifacts are prevented.